Summary of Changes

Hello @ShenQianli, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request enhances the BOTS framework by adding an optional feature for collecting and integrating reference model evaluation results. It provides a structured way to evaluate a specified reference model against a dataset, log the performance metrics, and then merge these metrics back into the original dataset as a new feature. This allows for more comprehensive analysis and utilization of reference model performance within the BOTS ecosystem.

Highlights

• Reference Evaluation Workflow: Introduced a new workflow for collecting reference evaluation results for BOTS, allowing users to evaluate a reference model and integrate its performance metrics.
• Data Path Configuration: Updated bots.yaml and random.yaml to use manually configurable data paths, replacing previous environment variable placeholders for clarity and user control.
• Evaluation Result Integration Script: Added a new Python script (ref_eval_collect.py) designed to integrate collected evaluation results into existing datasets, adding them as a new column.
• Documentation Update: Updated both English and Chinese README.md files with comprehensive instructions on how to set up and run the new reference evaluation process.
• New Workflow Class: Implemented BOTSRefEvalCollectMathBoxedWorkflow to handle the logging of detailed evaluation results (model version, rewards, question, truth) to a JSONL file, ensuring safe concurrent writes with file locking.